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Abstract 



We study the transition towards effective payoffs in tlie prisoner's dilemma game 

O ' on scale-free networks by introducing a normalization parameter guiding the sys- 

c/3 ■ tem from accumulated payoffs to payoffs normalized with the connectivity of each 

^~*, agent. We show that during this transition the heterogeneity-based ability of scale- 

O ' free networks to facilitate cooperative behavior deteriorates continuously, eventually 

collapsing with the results obtained on regular graphs. The strategy donations and 

adaptation probabilities of agents with different connectivities are studied. Results 

K*" I reveal that strategies generally spread from agents with larger towards agents with 

_i ■ smaller degree. However, this strategy adoption flow reverses sharply in the fully 

f^ . normalized payoff limit. Surprisingly, cooper ators occupy the hubs even if the aver- 

'^ I aged cooperation level due to partly normalized payoffs is moderate. 

T-H ■ 
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1 Introduction 



The welfare of a society often demands unselfish behavior of their members. 
Although working for the common good via cooperation seems a reasonable 
demand in return for a stable and flourishing society, it is often fundamentally 
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at odds with the Darwinian principles of evolution suggesting that individuals 
should act exclusively so to enhance their own prosperity. While understand- 
ing the evolution of cooperation among selfish individuals is a puzzle faced by 
scientists across many different fields of research |Tj, the mathematical frame- 
work of choice for addressing the challenge is often the same, namely the evo- 
lutionary theory of games [2][3]IHl5] . Particularly the prisoner's dilemma game, 
consisting of cooperators and defectors as the two possible strategies, is consid- 
ered a paradigm for studying the emergence of cooperation between unrelated 
and selfish individuals [6]. The game promises a defecting individual the high- 
est income if facing a cooperator. On the other hand, the cumulative income of 
two cooperators is higher than that of a cooperator-defector pair, and higher 
still than that of two defectors. According to the fundamental principles of 
Darwinian selection individuals should decide to defect, which ultimately re- 
sults in social poverty. This unadorned scenario is indeed described by the 
classical well- mixed prisoner's dilemma game [5], where cooperators always 
die out. However, since cooperative behavior is an underpinning for a success- 
ful society, and is thus arguably present in everyday hfe [TfS] , the gap between 
the outlined theory and practice is obvious, indeed requiring refinement of the 
former. 

An important milestone in bridging the gap between the outcomes of the well- 
mixed prisoner's dilemma game and reality was the introduction of spatial 
structure and nearest neighbor interactions pioneered by Nowak and May 
[H], which enabled the cooperators to form cluster on the spatial grid and 
so protect themselves against being exploited by defectors. The conditions 
of some biological experiments confirmed this conjecture [lOj. However, there 
also exist compelling evidences that spatial structure may not necessarily favor 
cooperation p!T]. Albeit a statement issued for the snowdrift game, in view of 
difficulties associated with the payoff rankings in experimental and field work 
pi^fTB] , it certainly carries an important message and dictates that the search 
for additional cooperation-facilitating mechanisms is necessary. 

Indeed, research performed within recent years has made it clear that, besides 
spatial extensions and mechanisms such as kin selection, reciprocity, strategic 
complexity [2] , or asymmetry of learning and teaching activities [IS] , the spe- 
cific topology of agents defining the interactions among them plays a key role in 
determining the outcome of the evolutionary process [T6|17|18|19|20|21ll22f23f24 
Perhaps most prominently, scale- free (SF) networks [27], previously being 
identified as omnipresent in man made and natural systems [28] , have been es- 
tablished as extremely potent promoters of cooperation in both the prisoner's 
dilemma as well as the snowdrift game [T7fT^ . In contrast, some works have 
shown that the ability of SF networks to promote cooperation disappears 
if we apply averaged instead of the more widely used accumulated payoffs 
[29|30f31ll32] . Moreover, the interplay between the evolution of cooperation as 
well as that of the network defining the interactions among agents has also 



received considerable attention [33f34f35ll36f37f38j . showing specifically that 
the evolution of the interaction network might have a beneficial effect on co- 
operation, and finally, a widely applicable and simple rule for the evolution 
of cooperation on graphs and social networks has been proposed |3S]- For a 
comprehensive review of the field of research see [lOj . 



In this paper, we wish to elaborate on the prominent role of scale-free net- 
works and their ability to promote cooperation in the prisoner's dilemma game. 
More precisely, we study the transition towards effective payoffs in the pris- 
oner's dilemma game on SF networks, whereby the latter is realized via the 
normalization of the payoffs of each agent with its connectivity. In particular, 
we introduce a normalization parameter that guides the system continuously 
from non-normalized, i.e. absolute, to normalized, i.e. effective, payoffs. A 
similar but technically different interpolation formula was recently proposed 
also by Tomassini et. al. [5U]. However, our goal is not just to demonstrate 
deteriorated cooperator successfulness due to payoff normalization, but also to 
investigate the activity patterns of players that characterize the correspond- 
ing stationary states. For this purpose, we introduce strategy donation and 
adaptation probabilities of agents with different connectivities. These quanti- 
ties enable us to rigorously identify the direction of the strategy flow. In other 
words, they serve to explore typical sources and targets characterizing the 
microscopic strategy adaptation process in the stationary state of the game. 
We find that cooperator densities of different classes of agents show a relevant 
dependence on their degree span, thus segregating the agents in terms of the 
underlying network topology. Moreover, we also study the global intensity of 
the adaptation process and reveal a resonant-type behavior depending on the 
payoff normalization parameter. 

The paper is structured as follows. Section 2 is devoted to the description of 
the prisoner's dilemma game and the summarization of the main mechanism 
leading to an enhanced cooperation on SF networks. Also, it features the 
introduction of the normalization parameter that is being used to interpolate 
between the absolute and normalized payoffs. In Section 3 we present the 
results and discuss biological implications of our findings, whereas in the last 
Section we summarize our findings and conclude the paper. 



2 The Game 



We consider an evolutionary two-strategy prisoner's dilemma game with agents 
located on vertices of a scale-free network generated via the celebrated mech- 
anism of growth and preferential attachment [27], yielding a scale-free dis- 
tribution of their connectivity k. Initially, each vertex x is designated as a 
cooperator or defector with equal probability. Next, among all k^ neighbors 



of a randomly chosen vertex x one neighbor y is also chosen at random, and 
both play one round of the prisoner's dilemma game with all their neighbors 
fcj- and ky, respectively. Their accumulated payoffs resulting from k^ and ky 
interactions are stored in P^ and Py. Thereby, the payoffs are determined in 
accordance with the standard prisoner's dilemma payoff matrix having temp- 
tation b, reward 1, and both punishment as well as the suckers payoff 0, where 
1 < 6 < 2 to ensure a proper payoff ranking. Crucially, before attempting 
strategy adoptions, the two accumulated payoffs are normalized according to 

Ti = aPi + (1 - a)^ (1) 



where i = x or y and the newly introduced normalization parameter a, oc- 
cupying any value from the unit interval, determines exactly to what extent 
the payoffs are normalized with the number of interactions from which they 
were obtained. Evidently, a = dictates the use of fully normahzed, so-called 
effective payoffs, whereas a = 1 utilizes absolute, i.e. accumulated, payoffs. 
Finally, in the spirit of the numerical implementation of the replicator dy- 
namics, only if Py > Px agent x adopts the strategy of y with probability 
{Py — Px)/[{)- — (y)h + abkm], where km is the largest of k^ and ky. Since the 
strategy adoption is attempted only if Py > P^ this probability is always be- 
tween and 1, thus constituting the uncertainty related to each attempt of the 
strategy adoption. An elementary part of the Monte Carlo (MC) algorithm, 
identical to the one often employed in population genetics, is thus given by 
first randomly choosing a pair of players and letting them both play the game 
with all their neighbors, and second, by attempting the strategy adoption only 
if Py > Px. In accordance with the random sequential update, each individ- 
ual is selected once on average during a particular Monte Carlo step (MCS), 
which consists of repeating the elementary part N times corresponding to 
all X = 1, . . . , A^ vertices. Our simulations were performed on populations of 
A^ = 5 ■ 10^ to 5 • 10^ agents, whereby the average connectivity of the scale-free 
network was always ^ = 4 as a result of the graph-generating algorithm [27] . 
The application of the asynchronous MC update scheme diminishes the depen- 
dence of the stationary state on the initial distribution of the two strategies 
and the realization of the generated SF network [3PJ. Characteristic quantities, 
such as the equilibrium frequencies of cooperators pc or the strategy adap- 
tation probabilities defined below, were evaluated after sufficient transients 
ttr ~ 2 ■ 10^ to 10^ MCS where the sampling period was two times longer than 
ttr. All runs were repeated on 10 to 30 independently generated graphs. 

We stress that our study focuses on the dynamical features of the game in 
its stationary state because the key mechanism leading to an enhanced co- 
operation during transient periods has already been uncovered in previous 
works ^40,.41j . The most important topological feature of a SF network that 
promotes cooperation is the high connectedness of the hubs. Starting from 
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Fig. 1. Temporal evolution of the cooperator density towards its stationary state 
for b = 1.5 and a = 1. The arrow marks the time when the last defector hub was 
overtaken by a cooperator hub. After this event the whole defecting neighborhood of 
the defeated hub imitated the change, resulting in an enhanced level of cooperation. 

a random distribution of strategies, all hubs can win over the neighbors be- 
cause their hub-specific high connectivity results in large cumulative payoffs. 
Therefore the players that are linked to a hub will imitate its strategy, which 
eventually results in a cloud of homogeneous strategists around each hub |42j . 
In the long run, such an evolutionary process strengthens the cooperative 
and weakens the defector hubs. In particular, when two competing hubs play, 
which happens statistically later due to a small probability of such an event, 
the cooperating hub wins [5]]. Shortly afterwards, the whole cloud of defectors 
that was previously supported by the defeated defector hub adopts the coop- 
erative strategy as well, which ultimately results in a substantial increase of 
cooperator density on the network. This mechanism is demonstrated in Fig. [H 
where the density of cooperators is plotted as a function of time at b = 1.5 
warranting a high level of cooperation in the stationary state. At around 16000 
MCS (denoted by an arrow) the last defector hub changes its strategy, which 
results in a rapid increase of pc because shortly afterwards the whole cloud 
of defectors gives in to the cooperative strategy as well. The resulting state 
proves to be stable, bounding fluctuations of pc around a fixed steady-state 
value. 

Before we turn to examining the results, some remarks regarding the motiva- 
tion behind the introduction of the normalization parameter a are in order. In 
particular, while effective payoffs allow for a better mathematical study of the 
dynamics and system-size effects, as well as easier comparisons with analytical 
treatments both at the mean-field and pair-approximation level, on the other 
hand, absolute payoffs lead to a more exciting interplay of the two strategies 



as then cooperators can dominate not only by clustering (as on regular grids) 
but also via the benefits offered by the underlying SF interaction network. 
The dichotomy of results obtained via effective {a = 0) and absolute {a = 1) 
payoffs dictates the necessity for studying the behavior in-between the two 
extremes, and the explicit use of the interpolation formula given by Eq. ([1]) 
enables an elegant accomplishment of this task. 



3 Results 



We start by examining the impact of a on the equilibrium frequencies of coop- 
erators and defectors. Figure [2] summarizes the results, from which it becomes 
instantly obvious that as a transits from 1 towards 0, thus gradually introduc- 
ing the effective payoffs into the strategy adoption of the employed prisoner's 
dilemma game, the cooperative behavior suffers continuously. In particular, 
while the absolute payoffs (a = 1) are able to sustain the domination of co- 
operation throughout the whole range of b, the effective payoffs {a = 0) prac- 
tically induce an extinction of the cooperative trait for all b > 1.2. In fact, 
results for a = are quite similar to the ones obtained for regular graphs with 
nearest neighbor interactions [17], thus suggesting that the effective payoffs 
eliminate the advantage of cooperators given to them by the heterogeneity of 
the SF network [31J. We should mention that in [30J a qualitatively identical 
result was reported, albeit by using a slightly different interpolation formula. 

Although it seems very inviting to conclude that the normalization of pay- 
offs via a simply eliminates the advantage of cooperator hubs given by their 
large number of links, arguably found crucial for the dissemination of coop- 
erative behavior pT|^ , the dynamics behind results presented in Fig. [2] is 
in fact more subtle. To elaborate on this fact we first introduce the quantity 
X = Pel/ Pc^ where pd is the fraction of directed links ending at a cooperator. 
Although Pel ~ pc by absorbing states, this is not necessarily true by mixed 
states. In particular, we argue that by mixed states x provides complementary 
information about which strategy occupies the main hubs of the SF network. 
More precisely, pd > pc suggests that cooperators are residing on vertices 
with a larger number of links than defectors. Thus, values of x > 1 are a 
sufficient condition for the fact that hubs are predominantly occupied by the 
cooperative strategy. On the other hand, x = 1 indicates that the system is 
either in an absorbing state (which is trivially true), or more interestingly, that 
neither of the two strategies succeed in predominantly occupying the hubs of 
the network by mixed states. Results in Fig. [3] confirm this interpretation of x 
since the value stays close to 1 as long as the stationary state remains predom- 
inantly cooperative. Strikingly though, x raises when pc decreases, signaling 
that the hubs remain predominantly occupied by cooperators despite the fact 
that values of a close to introduce effective payoffs that diminish the advan- 
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Fig. 2. Fraction of cooperators pc in dependence on h and the normalization pa- 
rameter a. Evidently, pc decreases continuously for all values of 6 as a tends to 0. 
Employed parameter values were: A^ = 10^, ttr = 10^, and the results were averaged 
over 30 different realizations of the underlying scale-free network. 



tage of multiple connections. Naturally, when the majority of agents adopts 
the defecting strategy the hubs change their strategy as well, and thus x again 
quickly converges to 1. Nonetheless, results for values of a close to reveal 
that hubs are occupied by cooperators despite the apparent dismissal of their 
advantage that is implied by the normalization of the accumulated payoffs 
with the number of interaction from which they were obtained |43j . 



Our conjecture can be additionally strengthened by studying pc for different 
classes of agents depending on their connectivity k. For this purpose, we divide 
the agents into three categories such that the connectivity span belonging to 
each group is equally large on the logarithmic scale. The fraction of cooperators 
in the group with the largest, medium, and the smallest connectivity is denoted 
by p^, Pq , and pf., respectively. Results presented in Fig. H] clearly show that 
the main hubs are actually the ones that remain occupied by cooperators the 
longest as a decreases towards 0, while in fact pff , and even more so p^, 
start to deteriorate substantially sooner. In sum, results presented in Figs. [3] 
and m clearly demonstrate that, in contradiction with the intuitive reasoning, 
hubs remain occupied by cooperators until the actual a = limit is reached, 
despite the fact that even for values of a close to virtually effective payoffs 
are governing the strategy adoption process of the game. In other words, even a 
tiny benefit originating from the larger number of links suffices for cooperators 
to invade the hubs. 
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Fig. 3. Ratio x between the fractions of links that point to cooperators pd and 
nodes occupied by cooperators pc in dependence on the normahzation parameter 
ahy b = 1.5 (dashed hne). Sohd hne shows the decrease of the level of cooperation. 
These results suggest that hubs remain occupied by cooperators almost for all values 
of a. 
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Fig. 4. Fractions of nodes that are occupied by cooperators, depending on their 
connectivity A;, as a function of a by b = 1.5. Results are presented separately for 
three connectivity classes, later being small (S - dotted line), medium (M - dashed 
line), and large (L - solid line). Cumulative values of pc displayed in Fig. [3] are close 
to the values depicted by the S curve. 
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Fig. 5. The probability of strategy donations P^ (solid lines) and adaptations Pa 
(dashed lines) in dependence on the connectivity of agents k. Values on the horizon- 
tal axis, running from 1 to 100, correspond to the minimal and maximal connectiv- 
ity, respectively. The steady-state activity patterns are shown for four representative 
values of a, all obtained by 6 = 1.5. 
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Fig. 6. Comparative plots of strategy adaptation probabilities of agents with degree 
k by different values of a. Curves characterize possible targets of strategy adapta- 
tion. 

To get a deeper understanding of the presented results, we further investi- 
gate the microscopic mechanism behind the deterioration of the cooperation- 
facilitating effect of SF networks as a -^ 0. Therefore, we study the strategy 
adoption process of agents with different connectivity, thereby providing in- 
sights into the dynamics leading to the characteristic properties of stationary 
states. For this purpose we calculate the probability Pa{k) that an agent with 
connectivity k will adopt the strategy of its neighbor. This quantity character- 
izes the potential targets of the invasion process. To identify possible sources 
we also introduce Pd{k), which is the probability that an agent with connec- 
tivity k will pass (donate) the strategy to its neighbor during an elementary 
process. Figure [5] illustrates the introduced probability distributions for dif- 
ferent values of a. In case of accumulated payoffs (a = 1) the strategies of 
agents with midddle and high connectivity are practically frozen. They can- 
not change but are only able to spread their strategies to agents that have 
fewer links. In the opposite limit (a = 0), when effective payoffs apply, only 
agents with small connectivity can donate their strategy to other agents. The 
comparisons of Pa and Pd distributions reveal that the direction of strategy 
flow remains unidirectional for all a > 0. The only exception thereby is the 
a = case. In sum, strategies generally spread from agents with larger to 
agents with smaller connectivity, and moreover, this strategy adoption flow 
reverses sharply only in the fully normalized payoff limit. 

The comparison of "target distributions" obtained by different values of a 
presented in Fig. [6] reveals that for a close to hubs change their strategy 
far more often than their counterparts by larger values of a. As a increases 
towards 1, the bulk of adaptation activity shifts from agents with larger to 
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Fig. 7. Comparative plots of strategy donation probabilities of agents with degree k 
by different values of a. Curves characterize possible sources of strategy adaptation. 

agents with smaller connectivity via an intermediate non-monotonous phase. 
We argue that the mutable nature of cooperator hubs, observed for close to 
zero values of a, is the main reason for their inability to spread cooperative 
behavior, despite the fact that, as suggested by results in Figs. [3] and HI the 
majority of time defectors fail to conquer this prime spots of the network. 
Thus, the main mechanism behind the destructive impact of effective payoffs 
on the cooperative strategy is not the inability of cooperators to occupy the 
main hubs of the network, but rather their inability to permanently sustain 
that spots by comparatively (in comparison to results obtained for values of 
a close to 1) frequently losing their battles to the defective intruders. 

The target and source distributions presented in Figs. E] and [7] also reveal a 
resonant-type increase of the overall adaptation process as we change the value 
of a from 1 to 0. This non- monotonous curve, not shown here, can be well 
approximated in the mean-field level by the product of the two strategy den- 
sities PcPd- Consequently, in the coexistence phase the stationary state is not 
rigid but intensely fluctuates due to frequent strategy adaptations, especially 
when pc is far from the absorbing state. 



4 Summary 



In sum, we elaborate on the prominent role of scale-free networks and their 
ability to promote cooperation in the prisoner's dilemma game. We study the 
transition towards effective payoffs that is driven via a single normalization pa- 
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raineter a. We find that the main mechanism behind the destructive impact of 
effective payoffs on the cooperative strategy is not the inabihty of cooperators 
to occupy the main hubs of the network, but rather their inabihty to perma- 
nently sustain that spots by comparatively frequently losing their battles to 
the defective intruders. In case of normalized payoffs players having the small- 
est connectivity rule the evolution. The careful analysis of strategy donation 
and adaptation habits reveals that the flow of strategy transmissions reverses 
to the opposite direction instantly even if the payoffs governing the evolution 
shift only slightly from the effective towards the absolute case. We find also 
that the heterogeneous host network implies heterogeneity of the distribution 
of cooperators on nodes that have different connectivity, and moreover, that 
cooperators always prefer widely connected nodes (hubs) irrespectively of the 
level of payoff efficiency. 
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